Forward-backward Correlation for Template-based Tracking

A significant limitation of traditional template-based trackers is their inability to handle out-of-plane rotation, which can cause total self-occlusion of the target. We present a simple extension to template-based tracking that overcomes this problem. A forward correlation-based search for computing the transformation (displacement and scale) between two image frames is augmented with a backward correlation-based search for grouping the pixels with similar image velocities. In effect, the latter performs motion segmentation on the pixels around the target to automatically update the model as it changes over time, while avoiding drift. A gradient module assists the algorithm when the background contains little texture. Experimental results demonstrate the effectiveness of the technique in both textured and untextured environments, with complete self-occlusion caused by full 360-degree out-of-plane rotation, along with scale changes.